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? 3 10 BACKGROUND- FIELD OF THE INVENTION 

in 

III 

i j This invention generally relates to a panoramic image reproduction system, and in 

f i particular to the use of rotary scanning elements in a camera design. This invention also 

1 LI relates to the use of aerodynamic elements to reduce the apparent weight and provide 

12 1 5 other benefits to a rotating apparatus, and to the use of gyroscopic stabilization of a 
camera platform. 

DESCRIPTION OF THE PRIOR ART 

20 Cameras for capturing still or motion pictures of the external world have usually used an 
area array sensor, such as a 640x480 Charge-Coupled Device (CCD) for NTSC video 
images. These area sensors use a rectangular or square area of light-sensitive elements 
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and expose them all at once. New forms of Complementary Metal Oxide Semiconductor 
(CMOS) sensors for cameras are also being made in a variety of resolutions. 

Linear scan sensors are also being used to increase resolution for specific applications. 
5 These sensors have a single row of sensors, designed for monochrome or color readout, 
and sensitive to either visible light or some other part of the spectrum such as infrared or 
ultraviolet. The image is made by successive readouts of the sensor as it moves across 
the image, building up the final resolution. This is also sometime referred to as a slit scan 
or line scan sensor. Digital scans of this type are already found in such devices as fax 

10 machines and flatbed scanners. Line scan cameras are often used in industrial processes 
to obtain clear pictures of objects in motion. Examples of these cameras are the Wintriss 
Engineering OPSIS 5150ALC for monochrome images, which uses a monochrome CCD 
linear sensor with 5150 pixels and onboard image processing, and the DALSA CT-E4- 
2048 W, which uses a color CCD linear sensor with 2048 pixels. Typically these cameras 

15 are fixed in place, and look at objects moving past them. Unlike area-scan sensors, the 
resulting images do not suffer from motion blur. 

Scan-back digital cameras, such as the Better Light Model 8000 camera back are also 
doing high-quality digital still photography. The use of a slow scan from a linear sensor 
20 allows for maximum resolution to be built up when photographing artwork, documents or 
other subjects. 
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Scanning sensors are also used in so-called "push-broom" satellite surveillance 
applications, where as the satellite sweeps across the surface, a succession of readouts of 
the scanner creates a very large and detailed overall image. Existing panoramic slit scan 
cameras have also been made that use a moving slit to expose a strip of film, such as the 
5 Widelux™ camera for stills. 

Various patents describe scanning cameras. All typically feature a rotation of the scanner 
to create the image according to an optical axis radial to the axis of rotation. Keller's 

U 

.ail 

U "Panoramic Camera" (US Patent 5659804), for example, describes rotation of a scanner 

t.i 

Ifi 10 where the nodal point of the optics for the scanner is located radially according to the axis 

•fin 

f ; 1 of rotation. OxaaP s "Method and Apparatus for Producing a 360° Spherical Visual Data 

M 

L Set" (US Patent 5903782) also involves a radial optical axis, this time for a "fisheye" 

KI 

p lens. Jehle's "360 degree IR Surveillance with Panoramic Display" (US Patent 4977323) 

j £j uses an infrared scanner also aligned radially to the axis of rotation. Federals "Line- 

d 

1 5 Scan Panoramic Camera" (UP Patent 4532544) also describes a radial axis for a 
television camera. And Globus et al's "Panoramic Camera" (US Patent 4241985), 
marketed as the Globuscope™, employs a slit and a radial optical axis to expose a film 
strip. Other film strip-based scanning cameras have also been made, going back to the 
cumbersome Cirkut camera of the nineteenth century. Contemporary examples include 

20 the Widelux, which exposes a film strip in 120-degree increments of rotation, and the 
Hulcherama camera which exposes a continuous strip of film in any amount of rotation, 
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using gear motors to advance the film in time with the sweep of the lens. Woltz's 
"Panoramic Motion Picture Camera and Method" (US Patent 4602857) describes a 
rotating motion picture film camera. All of these cameras share the same orientation of 
the optical axis as being radial to the axis of rotation. 

5 

What these solutions lack is an effective way to record not only the appearance of the real 
world in motion through the slit, but also to use that information to create a stereoscopic 
image of the entire field of view. Without that stereoscopic information, the most 
realistic picture of a surrounding scene is not possible, as well as other applications such 

10 as rangefinding that depend on parallax offsets between different points of view. The 
reason for this is that existing rotary solutions assume an optical axis for the light path 
that is radial to the axis of the rotating array. This restriction precludes the use of other 
axes in a rotating array for stereoscopic effects or increased resolution. If one attempted 
to create a stereoscopic view with two panoramic scans taken with such cameras, by 

15 taking them from two laterally offset points in space, the stereo illusion would be strong 
only for the area directly orthogonal to the axis connecting the two points, but would fall 
off to zero as you approached that axis in the course of looking around. This is described 
in McMillan & Bishop's "Plenoptic Modelling: An image-Based Rendering System" in 
SIGGRAPH 95 Conference Proceedings pp. 39-46. An overview of stereoscopic 

20 imaging systems, including relevant US and foreign patents, can be found in Michael 
Starks' summary article "Stereoscopic Imaging Technology" from 3DTV Corporation 
(http://www.3dmagic.com). 
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Existing designs for rotary scanning cameras also do not address other important 
considerations of design needed for successful use of the camera in the field. These 
include the noise that the camera makes, which affects whether sound can be recorded 
5 effectively together with the picture, the weight of the unit, which affects whether a 
camera operator can carry it, and the stability of the camera, which is important for 
steady, natural motion pictures when the camera moves through a scene in the course of 
recording. 



10 The present invention will address all of these shortcomings in the prior art. The present 
application is based in part on the disclosure document no. 466771 "Rotating Scan 
Camera with Oblique Axis" filed Dec. 22, 1999. 
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OBJECTS AND ADVANTAGES OF THE INVENTION 

A. It is an object of the present invention to provide a method, and an apparatus for 
applying that method, for stereoscopic imaging of a surrounding scene by means of 
5 separated views having equal parallax separation throughout the scene. The stereoscopic 
panoramic imaging method of the present invention is not limited to an optical axis radial 
to any axis of rotation, unlike the prior art. 

q 

jlj B. It is also an object of the present invention to provide a method, and an apparatus for 

f 1 10 applying that method, for increasing the resolution of the recorded image by the use of 
P interpolated and multiplexed line scan sensors. Unlike the prior art, which is directed 

I 

L toward single sensors and optical paths, this use of multiple sensors enables better 

O 

[ Jf resolution in the final image and the use of sensors that would otherwise be unable to 

J' j give good results in a single scan. 

n 

15 

C. It is also the object of the present invention to provide a method, and an apparatus for 
applying that method, for recording panoramic images in full motion, by means of a 
simplified recording mechanism. The prior art is largely directed toward the production 
of panoramic still photographs, and the art featuring film-based recording in particular is 
20 unsuitable for motion picture photography, because of the need to put a large and heavy 
film transport and storage mechanism into rapid motion. 
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D. It is also an object of the present invention to provide means for recording a fully 
spherical field of view in a seamless manner. The prior art for panoramic scanning is 
primarily directed toward a cylindrical panoramic format that omits a substantial portion 
of the surrounding scene. Spherical imaging to date has involved multiple views that 

5 must be seamed together prior to viewing. 

E. It is also an object of the invention to reduce drag and noise due to the movement of 
lenses. Streamlined front lens elements are unknown to prior art. 



elimination of external gyroscopes, a lighter apparent weight due to aerodynamic lift, and 
greater axial stability due to higher angular momentum relative to the overall weight of 
the camera. 



G. It is also an object of the invention to dynamically adjust the convergence of a 
panoramic stereoscopic view in order to to focus viewing on objects at selected distances. 
No prior art addresses this problem. 

20 H. It is also an object of the present invention to provide a convenient and cost-effective 
method and apparatus for recording high-quality scan image data on film. No prior art 
has been able to accomplish this. 




10 F. It is also an object of this invention to improve the stability of a camera by using 



rotation. The advantage of the present invention over the SteadiCam approach is the 
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I. It is also an object of the present invention to enable smooth and balanced operation of 
a rapidly rotating camera using internal film reels. The prior art does not address the 
optimal arrangement for motion and stereoscopic recording. 



FIG. 1 shows an oblique view of a preferred embodiment featuring a rotating drum, the 
motor and carrying handle for it, and two sensors in offset positions, showing an aperture 
10 slit for each sensor. 

FIG. 2 shows a top view of the rotating drum of the preferred emodiment, showing the 
sensor and lens components of the subcamera assemblies, along with the optical axis 
arrangements characteristic of the system. 

FIG. 3 shows a closeup schematic top view of the preferred embodiment using a fisheye 
1 5 lens with a streamlining covering for the drum surface. 

FIG. 4 shows an oblique view of the camera featuring the preferred use of gyroscopic 
elements to improve the stability of the camera. 

FIG. 5 is a top view of the camera drum in the preferred embodiment, showing the action 
of the spinning drum's sensors where an object in the field of view is scanned from 
20 different points of view, allowing for stereoscopic effects, with the addition of a 
rangefinding device as part of the scanning drum. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
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FIG. 6 shows an oblique view of the preferred embodiment of the rotating camera, with a 
hemispheric field of view for each of two sensors in a rotating camera drum with 
conventional fisheye lenses and streamlining elements. 

FIG. 7 shows two equirectangular stereoscopic views produced by scans of the preferred 
5 full spherical field of view according to the present invention. 

FIG. 8 is a schematic view showing the application of lateral distortion to portions of a 
recorded image to achieve stereoscopic convergence at a given distance, such as in would 
be produced by delays in the timing of the recording of scans. 

FIG. 9 is a schematic view showing a preferred embodiment wherein lateral distortion is 
10 applied to a region of interest within the overall image to achieve stereoscopic 
convergence 

FIG. 10 shows a block diagram of the various components of the entire system in the 
preferred embodiment. 

FIG. 1 1 shows a top view of an alternate arrangement for increasing the overall 
15 resolution, here with four sensors in place in a rotating drum, spaced around its 
circumference. 

FIG. 12 shows a schematic diagram of the adjacent recording of the scans from different 
sensors, using various amounts of delay for the signal from each sensor. 
FIG. 13 shows a closeup schematic top view of a conventional lens for the sensor, and 
20 ways to streamline the rapidly rotating drum surface. 

FIG. 14 shows an exploded oblique view of the camera showing the alternate addition of 
aerodynamic elements to reduce its apparent weight 



FIG. 15 shows an oblique schematic view of an alternate external film-based recorder for 
camera image information. 

FIG. 16 shows an oblique view of an alternate embodiment of the camera drum using a 
film-based recording system. 
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DETAILED DESCRIPTION 



A rotary scanning camera of a novel design offers improved performance in capturing a 
5 panoramic surrounding scene. One or more line-scan sensors are used in a sweeping 
recording of a surrounding scene. The scan can be of a partial rotation or of one or more 
full rotations. Unlike the prior art, these sensors each have an optical axis that does not 

p| intersect the axis of rotation. 

v3 

ill 10 In the preferred embodiment, a rotating drum holds two line-scan devices for scanning 

O the appearance of a surrounding field of view from stereoscopically separated viewpoints, 

M 

6 \_, with each optical axis on approximately a tangent line to the cylinder. This creates a pair 

| *f of offset sweeping scans that record objects in the surrounding view from one, then the 

other point of view, thereby creating an effective parallax separation in the recorded 
15 images. This parallax separation is the same for every point in the rotation of the drum. 

r 

FIG. 1 shows an oblique view of a rotatable scan camera according to the present 
invention. The camera comprises a rotatable drum 2, actuated by a motor 4. This drum 
has a central axis of rotation and a perimeter, which is the surface of the drum farthest 
20 from the axis of rotation. Two linear sensors, a Right-hand sensor 6 and a Left-hand 
sensor 10, are disposed at the circumference of the rotatable drum 2, each sensor having 
an aperture slit 8, 12. Each linear sensor has an associated lens through which light 
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passes according to a central optical axis and forms a focused image on an image plane 
orthogonal to the optical axis. The linear sensor's active surface is located in this image 
plane. Each sensor also has associated electronics for the input and output of signals. 
The sensor, optics and electronics form a subcamera assembly within the rotatable drum, 
which rotates in a direction shown by the arrow, 14, preferably clockwise as seen from 
above. 

For recording a surrounding scene in motion, this drum should be rotating very quickly, 
and the scans should be as rapid as possible. A preferred embodiment for recording 
motion would be a 5-inch diameter drum rotating at 3600 RPM, with two sensors, with a 
complete scanned image produced by each sensor every rotation, for an effective frame 
rate per sensor of 60 frames per second. The resolution of the image is constrained only 
by the sensitivity of the sensor and its readout rate, and the ability of the data recorder to 
keep up. To achieve a pleasing level of resolution when viewing a closeup of a part of 
the image, as will commonly be done, then the resolution of the overall image should be 
as large as possible. A preferred resolution for the scanned image frame produced by 
each sensor would be 2,000 by 4,000 pixels, which represents the rotation of a 2,000- 
pixel sensor sampled 4,000 times during each rotation. If this 8 million-pixel image 
frame is produced 60 times per second, this means 480 million pixels produced per 
imager per second, times two channels, or close to a billion pixels per second. This level 
of resolution, together with the stereoscopic qualities of the image, represents an 
extremely high level of realism. 

12 




FIG. 2 shows a top view of the rotating drum 2, showing the preferred tangential optical 
axes 16 and 18 of the Right and Left sensors 6 and 10, and their associated lenses at 20 
and 22, respectively. A sensor such as at 6 is usually wider than the aperture slit 8; this 
5 means that effectively, because of the space constraints of the interior of the drum, an 
optical axis 16 cannot be actually on the tangent of the drum, but must be located a short 
distance away. What is shown here is an optimal arrangement, with two roughly parallel 
m axes 16^ 18 with apertures located slightly forward of the tangent point 24. Other axis 

j ; i arrangements can be used, except for a radial axis 30 intersecting the center point 26. 

ill 10 Examples are a convergent axis such as is shown at 34> or a divergent axis such as at 70. 

m 

b A true tangent axis is shown at 32. 



Any interruptions to the surface of the drum when it is rapid rotation will create noise or 



whistling sounds as the drum rotates. This sound could draw unwanted attention to the 

v. -J 

M 15 camera itself, or interfere with any sound recording that is taking place along with the 
recording of the picture, as is commonplace in motion picture production. Because the 
rotary panoramic camera of the present invention would see so much of the surrounding 
view, a microphone would have to be placed very near the camera, such as in a location 
directly below it, in order to be invisible. This makes it even more important that the 
20 operation of the camera be silent, so as not to transmit any camera noise to this nearby 
microphone. To minimize surface interruptions from the subcamera lenses, streamlining 
elements should be used. 
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FIG. 3 shows the preferred embodiment of a conventional fisheye lens 38 for a sensor 6, 
and an especially wide streamlining front element 40 conforming to the drum surface 42 
that takes into account the wide horizontal field of view characteristic of this type of lens. 

5 For a specialized line-scan lens, the lens is very narrow and flat, with an arrangement of 
elements like a slice taken through the middle of a conventional lens. This flat lens has 
the advantage of the least surface interruption, but lacks the light-gathering abilities of a 
more conventional round lens, which is the preferred embodiment. While a flat fisheye 
line-scan lens for a slit aperture, such as those produced by Coastal Optical, may have an 

10 f-number of f8, a conventional round fisheye lens could have an f-number of £2.8, for 
nearly eight times more light delivered to the sensor. 

The rapid rotation of the camera drum required for motion picture recording creates 
special challenges. This rapid rotation, however, also has certain advantages. This 
1 5 motion can be used to increase both the stability and the portability of the camera when it 
is use. 

FIG. 4 shows how increased stability in the camera, especially when it is carried by a 
camera operator, can come through the use of the gyroscopic effect. Unlike other 
20 scanning cameras, the optical and electronic elements in the present invention can be 
concentrated around the periphery of the drum, producing a weighted wheel 44 balanced 
around a central axis of rotation 46 and a relatively open center 48, which fits the design 
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of a classic gyroscope. The rapidly rotating drum 2 and its attached motor 4 thus form a 
unit that tends to maintain its position in space. The inertia of the rapidly spinning drum 
weight acts to maintain the wheel in its original plane and to resist attempts to turn it into 
other orientations. This produces a steadier camera that removes bumps and shakes from 
a recorded motion picture. Gyroscopically stabilized platforms are found in various 
fields, as well as in the Steadicam™ line of camera supports for camera operators. In the 
present invention, the gyroscope is integrated into the design of the camera itself. A 
camera operator is shown at 50, holding a handle 52 which in this case has an optional 
universal joint 54 at the top, to allow the rotating camera drum to float more freely in 
space. The slip ring connection for passing signals to and from the drum is shown at 56, 
and the input and output line for the camera unit is shown exiting the handle at 58. 

FIG. 5 shows the action of the spinning drum's sensors in the preferred embodiment 
where an object in the field of view is scanned from, one, then another, point of view. 
Five optical axes for the spinning sensor 10 on the drum 2 are shown at 60, 62, 64, 66, 
and 68, and how the scans of the sensors, especially for very near objects 70, 
simultaneously sweep different areas 72 and 74. Thus objects are scanned at different 
times, from different directions. For instance, a comparable area 76 of the near object 70 
is shown as being scanned by sensor 6 according to the optical axis 78 at the first position 
80, but the same target area is scanned at the fourth position 66 for the sensor 10. This 
scanning of objects from different points of view creates the parallax separations within 
the image that are the basis of any stereoscopic effect. The lateral location within the 
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scanned image is a function of time, because it is the point in time at which the image 
appeared in the aperture. The scanned area on the object is shown for the Right 6 sensor 
at 82 and the Left 10 sensor at 84, as well of the area 86 of the object 70 where the 
images from the two sensors overlap. 

5 

The amount of parallax separation in the image is determined by the distance between the 
camera and the object being viewed. A preferred way to determine this distance is 
through the addition of a rangefinding element to the rotating camera drum. This 
rangefinder is shown as being made up of an emitter element 88 and a sensor element 90. 

10 This rangefinder, either a miniature radar or infrared-based unit, takes readings either 
continuously or at intervals, to determine an average distance for the objects in the 
surrounding field of view during the course of a rotation, and the corresponding 
adjustment profile recommended. This convergence profile for the rotation can be used 
to determine the amounts of delay to apply to the scans prior to their being recorded, or to 

15 determine a later correction for convergence in the display of the stereoscopic images. 

FIG. 6 shows an oblique view of the preferred embodiment of the rotating camera, with a 
hemispheric field of view for each of two sensors 6 and 10 in a rotating camera drum 2 
with streamlining elements such as at 40. A fisheye lens 38 with an optical axis 16 is 
20 shown for sensor 6, producing a scan 92 with a vertical field of view which extends 180 
degrees. As the drum 2 rotates 14 around an axis of rotation 46, according to a motor 4, 
additional scans are produced, such as the second scan 94. The same is true for the scans 
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96 and 98 as recorded by the sensor 10. The vertical boundary of a scan's field of view is 
at 100. The signals from these scans, as well as any necessary input signals, pass through 
a slip ring connection 56 and through a connecting cable 58, shown here exiting at the 
base of a handle 52. 

5 

FIG. 7 shows examples of two equirectangular stereoscopic views 102 and 104 produced 
by scans of a full spherical field of view from two sensors in the preferred embodiment. 
The scene here is of a carnival midway, with a swing ride sending riders overhead. This 
type of image is the result of a linear fisheye sensor with a hemispheric vertical field of 
10 view being rotated through a full 360-degree rotation. The characteristic stretching of the 
image toward the poles is evident. The two stereoscopic views can be considered as a 
"left-eye view" and a "right-eye view". 

For simplicity, the optical axes of the two sensors in the camera should be in the same 
15 plane and essentially parallel, facing toward the same side of the drum, for the effect of 
convergence on objects at infinity. By the addition of delays in one or both channels of 
the stereo image, the appearance of an object can be shifted in the recording or playback 
in either direction by adjustment of the amount of delay applied to each vertical scan 
before is used. The amount of adjustment for this convergence depends on the distance 
20 to the desired object. 
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FIG. 8 is a schematic view showing of the application of lateral distortion to portions of a 
recorded image to achieve stereoscopic convergence at a given distance, such as in would 
be produced by delays in the timing of the recording of scans. What is shown here are 
the timings of the recordings of the scans, relative to an uncorrected timebase of 
5 recording which proceeds from left to right within the frame of information 106. All of 
the scan information proceeds through a FIFO delay buffer with a certain default delay. 
Adding to or subtracting from this default delay allows for shifts either forward or 
backward in time. An uncorrected region is shown at 108. A region where the scans are 
shifted to the right (forward in time) is at HO, and a region where they are shifted to the 
1 0 left (backward in time) is at U2 . One view may be affected in this way, or both may be 
used for a greater range of corrections. 

When the final image is viewed, either as a mono or stereo image, the preferred 
embodiment employs the extraction of a smaller movable window of interest as needed. 
1 5 This presents a more natural view of a portion of the overall scene, while allowing the 
freedom to look around in any direction. This extraction of a region of interest can be 
from either a live or prerecorded image stream. 

FIG. 9 is a schematic view showing of the preferred application of lateral distortion for 
20 convergence to a region of interest 114 within the overall image 106 in order to align 
objects atop one another. To conserve computational steps in this essentially digital 
process, this image transfoimation can be done only on the portion of the whole frame 
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within or immediately adjacent to the region of interest 114. An example of a portion of 
the scans shifted forward in time is at 116. and scans shifted backward in time are at 118 . 

The preferred recording device for picture information would be a digital recorder, such 
as a streaming tape or based digital recorder. Application of image compression to the 
picture information prior to use reduces the bandwidth necessary for any digital storage, 
transmission, or playback. 

FIG. 10 shows a block diagram of the various components of the entire system in the 
preferred embodiment. The Right 6and Left 10 Linear Sensors have equivalent 
associated components such as optics 20 and 22 and optical axes 16 and 18, respectively, 
forming two subcamera systems within the boundaries of the camera drum 120. Each 
sensor has its associated power and drive electronics 122 which are coupled to a central 
Power source 124 and Control electronics 126, which are realted to the motion control 
regulation 128 for the motor 4. The sensor outputs an electronic signal representing raw 
digital image data 130. This data goes through image compression 132 and any 
necessary I/O formatting 134. Then the image signal exits the drum through rotary Slip 
Rings 136. A separate rangefmder 138 on the drum outputs distance data 140 for 
controlling stereoscopic convergence. The image data from the sensors then is processed 
with the appropriate delays 142 for alignment of their scans together in time, plus any 
convergence adjustments. To conserve the number of recorders needed, the data from 
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both channels may be multiplexed together 144 prior to being sent to the recorder 146, 
which preferably should be digital. 

During the Playback process 148, the image data can be transmitted 150 prior to 
5 decompression 152 as part of a display process featuring the extraction of a movable 
Region of Interest 154 which appears on a Display 156 . 

Alternate embodiments of the present invention are directed toward the solution on 
specific problems. For example, line-scan sensors at the present time do not have a fast 

lit 10 enough readout rate to produce the above horizontal resolution if the drum spins around 

m 

s.,: i 

l!3 very rapidly. One solution to this would be to multiplex several sensors to fill in greater 

f,i 

11 resolution. 
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FIG. 1 1 shows an alternative embodiment of the present invention, comprising four 
1 5 sensors, 10, 16, 18 and 20, in place in a rotatable drum 2. If the requirement is for 4,000 
scans during one revolution, which represents one frame, and the fastest that could be 
read from an individual sensor is 1,000 readings per frame, then these four sensors 10^ 
158, 160 and 162 could be used to build up the proper resolution. Each of the four 
sensors has the same angle of optical axis 164, 166. 168 , and 170 relative to the drum 2, 
20 according to their associated lenses at 174, 176, 178, and 180, and the four sensors are 
equidistantly spaced around the circumference of the drum. In operation, with the drum 
rotating, each sensor would scan an object in the surrounding view at a slightly different 
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time. By the addition of digital delays, these scans can be recorded adjacently, to 
produce an apparently continuous image. 

FIG. 12 shows a schematic diagram of how a composite image is assembled from the 
5 data coming from the sensors, showing how sensor spacing about the circumference of 
the rotatable drum 2 is taken into account when recording signals from the sensors. At 
182 is a scan signal from the first sensor 10, and at 184 is the signal from the second 
sensor 158, taken at the times when they are at roughly the same point on the rotating 
drum relative to the surrounding scene. By adding an appropriate amount of fixed delay 
10 186 . these scans of a given portion of the surrounding field of view are recorded 
adjacently at 188 and 190 , respectively. Amounts of fixed delay for each sensor 
compensate for the time differences between when a scan took place and when it is 
recorded. It is preferable to have the sensor spacing be as close together as possible, to 
minimize any visible discontinuities due to these time differences between the adjacent 
15 scans as they are recorded in the final image. 

FIG. 13 shows an alternate embodiment for streamlining and soundproofing the camera. 
If a clear enclosure 192 is used to encase and soundproof the spinning drum 2, then the 
optical axis 16 of a sensor will exit the enclosure at a very oblique angle, producing 
20 distortion as it passes through the enclosure, and any seams in the clear enclosure will 
appear as interruptions in the overall recorded field of view. There is also the potential 
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for problems from internal reflections from the enclosure surface into the lenses. Thus an 
enclosure is not the preferred solution to this problem. 

The large front surface of a conventional lens 194 would oridinarily create an interruption 
on the drum surface 42 that would produce noise, but the front of the lens can be made to 
conform to the surface of the drum by a streamlining front element 196. An optimum 
design for a lens might be a hybrid, with a Gradium™ front element, where the differing 
refraction index characteristic of this type of glass runs laterally across the front of the 
lens, in a direction at a right angle to the line of the sensor. This produces a front element 
196 that has the optical characteristics of a flat element, but which is thicker on one side 
than the other, to match the contour of the drum surface 42. The slant could be reversed 
for opposite sides of the drum, again to match the surface. An alternate covering would 
be more conventional glass for streamlining, but this has the risk of introducing optical 
distortion in some form. 

The apparent weight of the camera when it is in operation is important to the camera 
operator, because the heavier the camera is, the harder it is to carry around Here again 
the rapidly spinning drum of the camera can be used to advantage. Added aerodynamic 
lifting elements attached to the rotating drum can offset the apparent weight, making the 
camera more portable. 
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FIG. 14 shows an alternate embodiment featuring the addition of aerodynamic elements. 
These aerodynamic elements can be one or more of a variety of types, depending on the 
amount of lift required and whether they must be relatively silent so as not to create noise 
that would interfere with any sound recording. These elements can be located in many 
5 different places on the rotating drum 2 and still remain invisible to the sensors. Because 
the center 48 of the drum can be relatively open, due to the peripheral positioning of the 
sensors and the gyroscopic distribution of the weight, a rotor or propeller 198 can be 

, r located even in the interior of the drum, as well as above 200 or below it 202. Surface 

U 

ift ripples 204 can also be built into the upper surface of the drum, to create a region of 

I ifj 10 lower air pressure and lift when the drum is spinning. Outer rotor flanges also in the 

m 

□ manner of propellers 206 can also be built into the outer surface of the drum, as long as 

II thev remain invisible to the sensors. All of these rotors can act to lift the camera upward, 

P 

! ^ and thus reduce the apparent weight to the camera operator. The camera operator's hand 

J-! should be shielded from the strong wind that these rotors would produce, and care should 

15 be given to the spinning tips of the rotors by adding guards such as at 208 if necessary to 
prevent them from causing injury. 



A special type of element can also be used for automatically keeping the front surface of 
a lens clean and dry, by making use of the speed of the relative air stream past the drum 
20 surface. For the 3600 RPM example cited above, the wind speed on the outer drum 

surface would be almost 55 mph. A special curved scoop channel enclosure 210, located 
beneath the lens but still invisible to it, can continuously draw in dry air from beneath the 
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drum and deflect it upward, creating a strong current of air that will deflect rain and dust 
away from the lens surface. Here a counterclockwise rotational direction is indicated at 
212 . and the central axis of rotation is at 214. 

5 For large amounts of image data, unusual recorders may be used. One applicable form of 
recorder is a film strip recorder. High-quality film has the advantage of offering higher 
density storage of image data, in an essentially analog form, at a lower cost than 
conventional digital streaming media, especially for large quantities of images. 

10 FIG. 15 shows an oblique schematic view of an alternative external film-based recorder 
for image information from the camera. Unlike conventional panoramic cameras, the 
film apparatus described here is located external to the camera drum itself. The 
dimensions of the film strip thus need have no physical relation to the primary drum 
apparatus. The only communication between the two is through electrical transmission 

15 of the signals from the sensors shown here coming in on a line 216. The image is formed 
for the film strip recorder by means that turn the electrical signals representing the scans 
back into light in a linear display 218. Amplification of the signals 220 can be applied if 
necessary to produce the proper image, which is focused and reduced through optics 222 
before it is recorded onto the continuously moving film at an exposure point 224. The 

20 supply reel for the film is at 226. and the takeup reel is at 228. Because the film strip is 
in constant motion, a continuous image is built up through the recording process. Any 
divisions in this recorded image on film, such as frame divisions, should be based on the 
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film sprocket holes outside the image, or other measurements of distance, and not be 
interruptions to the image. The image on the film can be digitized at another time if 
desired, such as for playback or transmission of the image. 

5 FIG. 16 shows an oblique view of an alternate embodiment of the camera drum using a 
film-based recording system. The entire apparatus must rotate according to an axis of 
rotation 230. The supply reel 232 is in an upper portion of the drum 234, and the film 
236 goes down into a middle portion 238 where it is exposed at 240 by a slit aperture 
according to the optical axis 16. The film proceeds dow n 242 to the lower portion 244 

10 where it is wound onto the takeup reel 246. A similar arrangement of parts would have to 
be done for a second point of view such as for stereoscopic recording. This arrangement 
has the advantage of offering balance of the weight of the supply and takeup reels around 
a central axis of rotation 214. This allows rapid rotation without undue vibration and 
instability. Film as a recording medium may be chosen because certain films may be 

15 more sensitive than a digital sensor, and yield better images. However, this approach has 
several disadvantages. Foremost among them are the size and weight of the apparatus 
required, and the difficulty of putting it into rapid rotation for scanning. 
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OPERATION, RAMIFICATIONS AND SCOPE 

Accordingly the reader will see that the rotating scan camera of this invention can be 
5 used to record a panoramic field of view with increased resolution and stereoscopic 
separation throughout the recorded image. In addition, the rotation of the camera drum 
can be used to impart stability and to lighten the apparent weight of the camera when it is 
in operation. 

10 Although the description above contains many specificities, these should not be construed 
as limiting the scope of the invention but as merely providing illustrations of some of the 
presently preferred embodiments of this invention. 

For example, a single oblique sensor will also produce a usable panoramic scanned 
1 5 image, although without the stereoscopic features of a dual scan. Lesser resolution 

sensors can be used for more inexpensive solutions or for situations where the bandwidth 
for recording or playback is limited. 



The arrangement of optical axes shown here as tangential can also be created within a 
20 larger drum. In this case, the axes are still parallel and in the same plane, but the 

apertures are located farther forward from the tangent point. Oblique but not tangential 
axes can also be used, either convergent or divergent relative to the tangential axis. The 
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optical axes in these cases still do not intersect at any point behind their lenses and within 
the drum, unlike the radial axis arrangement characteristic of the prior art. 

The sensors can be in monochrome, color, or having another spectral sensitivity such as 
Ultraviolet or Infrared An example of a color linear sensor is the Eastman Kodak KLI- 
21 13, a trilinear RGB sensor with a 2098x3 resolution, an image area of 29.4 x 0.24 mm, 
a sensitivity of 14 bits, and a readout rate of 60MHz. Other CCD and CMOS line scan 
sensors have other parameters. Generally speaking, at the present time, CCD sensors are 
more sensitive than CMOS sensors and allow for exposure of an entire vertical scan at 
once, but require more complex power sources. CMOS sensors, on the other hand, can 
incorporate many types of logic elements on the chips themselves. This enables on-chip 
functions such as digitizing, compression, enhancement, and pattern recognition. 

Preferably the line scan sensor should be as sensitive to light as possible, as well as 
having a very fast readout rate. The fast readout rate enables many individual vertical 
scans to be read from the sensor while it is in motion, and the sensitivity of the sensor 
enables a usable image to be produced even during the extremely short exposure time 
represented by such rapid scans. PixelVision and Scientific Imaging Technologies of 
Beaverton, Oregon have developed especially sensitive back-illuminated CCD's for 
astronomical use. 
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If multiple sensors must be used to build up the proper resolution, then they should be 
carefully spaced around the periphery of the drum while taking into account the desired 
resolution. For example, four sensors, each limited to a readout speed of only 1000 
scans during the course of a revolution, are used to build up an overall resolution of 4000 

5 scan totoal. If the four sensors are spaced within a span of 90 degrees on the drum and 
exactly 22.5 degrees apart, then every 62.5 scans (1000*(22.5/360), each sensor would 
reach the same position as the next one in line. The first sensor, in other words, would 
advance in the course of the revolution, and after 62.5 scans the next in line would reach 
the starting position of the first. What is needed, however, is not a mere duplication of 

1 0 the first reading, but another reading taken slightly ahead or behind this point, to help fill 
in the gaps represented by the readings of a single sensor. If the intention is to fill in the 
gap going backward, then the spacing between the sensors is increased by a slight amount 
representing one increment of the desired final scan resolution, in this case 4000 scans. 
For a 12.7 cm (5-inch) diameter drum, which is a preferred size to use because of the 

1 5 amount of parallax separation it produces, the circumference is 398.98mm or almost 
exactly 10 scans/mm as the drum rotates. Thus the added increment of spacing for each 
would be approximately one-tenth of a millimeter. 

Each sensor should have the same type of lens, to enable the matching of recorded 
20 images. Since the horizontal field of view of the line sensor is negligible, the important 
consideration for the lens is its vertical field of view. In the preferred embodiment, a 
fisheye lens, characterized by having a field of view of close to 180 degrees, is used to 

28 



m 
O 

iii 10 

m 



rn 

S X9 



achieve an essentially spherical recording of the surrounding view as the drum rotates. 
The lens should be small, of high quality, and fast, which means that it transmits light 
efficiently. Good quality lenses of this type are being made by Coastal Optical. Estelle's 
"Wide-angle Photographic Lens System and Photographic Camera" (US Patent 5502597) 
describes an especially compact and efficient design for a wide-angle lens. In order to 
ensure recording of the area directly above the rotating camera, while creating a "blind 
spot" directly underneath it to hide a camera operator or mount, it may be preferable in 
practice to construct the subcamera assembly so as to raise the axis of the lens by up to 10 
degrees from the horizontal plane of the drum. 



As the vertical angle of view increases, an exposure difference becomes apparent in 
different segments of the line sensor. For a global scan of a spherical field of view, for 
instance, this exposure difference comes from the different rates of speed found in 
scanning around the poles versus scanning at the equator. Another way to illustrate this 
15 is to observe the difference in rotational speed of a point on a globe near the poles, as 
compared to a point on the equator. To adjust for this, a fixed signal compensation 
function can be applied to the signals from different sections of the linear sensor. 

The motor assembly should be very smooth and quiet, and able to hold to a precise 
20 rotational speed Various types of electronically controlled motors have been developed 
to meet these characteristics. A recent line of direct-drive brushless dc motors from 
Thomson Airpax Mechatronics Group are 66% smaller and 30% more efficient than other 
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industry-standard motors. A smaller size for the motor would aid in the portability of the 
camera system by making it lighter in weight overall. Sinusoidal motion control is 
preferred for the smoothest possible regulation of speed. An 1800 RPM motor would 
produce an effective frame speed of 30 fps. This happens to be the type of motor used 
for the head drum in a VHS tape machine. 

Rotary couplings such as slip rings are used to transfer electrical signals to and from the 
electronic elements in the rotating drum. Litton Poly-Scientific's EC3848, for example is 
a high speed slip ring assembly of up to 10 lines for rotational speeds up to 10,000 RPM. 
Examples of the type of signals that would go into the drum are power and timing signals 
for the sensors, while the readout from the sensors would come out along one or more 
lines as required Some sensors have multi-tap readouts to increase effective resolution 
and data speed. The DALSA CT-E4-2048W, for example, uses the IT-PI-2048 sensor 
which has four simultaneous readout channels at speeds of 20Mz to 25Mz each, for an 
effective readout rate for the sensor of up to 100MHz. The number of rings in the 
coupling depends on the number of electrical channels needed to maintain proper 
connections. 

Rangefinding sensors come in a variety of configurations. The miniature radar-on-a-chip 
invented at Lawrence Livermore Laboratory in 1995 is one example of a low-cost sensor 
that could be used to obtain readings of the distance to various objects during the course 
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of a scan. Active Infrared Autofocus devices such are found on still cameras could also 
be used, especially to give distance readings within a range of around 20 feet (6m). 

For more convenient use of the image data from the sensors, compression should 
preferably be applied to the data prior to its transmission from the drum. Various 
methods for image compression are currently available, such as Discrete Cosine 
Transform (DCT) and the Motion Picture Experts Group (MPEG) family of motion 
picture compression. A preferred embodiment for compression in the present invention 
would be wavelet compression. The final data rate for the output would be determined 
by the desired level of quality in the image, as well as the optimum data rate of the 
recorder. 

For digital readings from the sensors, a variety of data transfer protocols could be used 
for sending the data for external recording, transmission or processing. The transmission 
can be in the form of a wired or, preferably, a wireless connection. The preferred 
embodiment makes use of one of the new high-quality high-bandwidth standardsThese 
include Fibre Channel (ANSI x3.230), which is made for fiber optics and has an effective 
speed limit of 400 mbps (million bits per second), the same speed as for the IEEE- 1394 
"Firewire" interface. Serial Digital Interface (SDI; a.k.a. SMPTE 292M) has a maximum 
speed of 1200 mbps, and Gigabit Ethernet, 1000 mbps. The Gigabit Internet standard 
(GVIF) has a maximum speed of 1776 mbps. An even faster data speed is represented by 
10 Gbps Ethernet standard now being developed by the 10 Gigabit Ethernet Alliance, 
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Ethernet Alliance, including such companies as Intel, Extreme Networks, Nortel, World 
Wide Packets, and other companies. This speed would handle the full-bandwidth data 
stream from the preferred resolution and frame speed described above as the preferred 
embodiment. 

The image data coming from each sensor represents a vertical column. The columns are 
put together to make the final image. Video images, on the other hand, are made up of 
rows, which are read out horizontally. In order to convert one to the other, an image 
made up of columns is read out in rows, which can be done easily using a frame buffer 
and a simple processor. A readout of an HDTV video image of approximately 1920 by 
1080 pixels (which can be identified as High-level, High-profile ADTV), is preferred, as 
a way of preserving maximum image quality within existing video standards. HDTV 
digital recorders such as the Panasonic AJ-HD2700 D-5 VCR (for a high-bandwidth SDI 
signal) or even the low-cost Panasonic PVHD 1000 D-VHS VCR (for a compressed 
signal) recorder are examples or recorders that could be used for this purpose. 

For a film strip recorder for the image information, the image can be formed for the film 
by a linear diplay. Many forms of displays and microdisplays may be suitable, depending 
on the brightness, size and image characteristics required. These displays include an 
LED or laser diode array, or a linear CRT, LCD, DLP (Digital Light Processing) or 
plasma display, with reduction and focusing optics as necessary to transfer the light onto 
the film. A linear color plasma display has the advantage of apparently continuous 



32 



example, has pixels measuring 1.08 mm square; therefore a 2098 pixel linear sensor 
requires an array 226.584 cm (89.2 inches) long, which requires substantial reduction 
optics. An alternate form of arrangement would be to split the display into a multiplicity 
of sections, which are combined to make a continuous recorded image. This can make 
use of any multi-tap readouts of the sensor, such as those featured in the DALSA camera 
mentioned above, where each of the four parallel channels coming from the sensor can be 
made to control its own display array. The Cine V Solitaire Image Recorder (US Patent 
4754334) is an example of an advanced film recorder for digital data. 

The extracted signals from the sensors can be sent to other locations for use via electronic 
transmission, for either a live or a prerecorded image. It is preferred that a movable 
region of interest be extracted from within the overall image, either live or prerecorded. 
If taken from the live image, then if desired, the rest of the image can be discarded. This 
has the advantage of reducing the overall requirements for transmission or processing. 

Thus the scope of the invention should be determined by the appended claims and their 
legal equivalents, rather than by the examples given. 
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ABSTRACT 

A scanning camera with a rotating drum has one or more sensors characterized by a non- 
5 radial optical axis. With two sensors on opposite sides of the drum and facing in 

substantially the same direction, stereoscopic recording of a panorama is accomplished as 
the drum rotates. Rapid rotation of the scanning camera produces panoramic motion 
picture recording, with the final frame speed dependent on the sensitivity and speed of 
the sensor, the resolution desired, and the capabilities of the recording device. The 

10 preferred embodiment employs rotating fisheye lenses for a substantially full-sphere field 
of view. Streamlining of the lens elements on the dram surface is described for quiet 
operation of the camera, even at high rotation speeds. The rapid rotation of the drum 
characteristic of motion picture frame rates can improve both the stability and portability 
of the camera. The gyroscopic effect of the rotating weight of the drum can increase the 

1 5 stability of the camera, and the apparent weight of the camera can be reduced by the 
lifting effect of aerodynamic elements such as rotors added to the rotating drum. The 
adjustment of convergence are described that improve the viewing of stereoscopic 
images. Additional sensors in the same arrangement are used to increase resolution 
through multiplexed recording of the image data. Recording image information using 

20 film, either internal or external to the camera drum, is also described as a cost-effective 
alternative to digital media storage. 
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LISTED PARTS IN DRAWINGS 

FIG. 1 

2. Cylindrical drum 

4. Motor for rotatable drum 
5 Right-hand sensor 

8. Aperture for Right sensor 

10. Left-hand sensor for stereoscopic scan 

12. Aperture for Left sensor 

14. Clockwise direction of rotation 
10 FIG. 2 

16, Semi-tangent optical axis for Right sensor 

18. Semi-tangent optical axis for Left sensor 

20. Right lens 

22. Left lens 
15 24. Tangent point 

26. Center of rotation 

28. Drum surface 

30. Radial axis 

32. Tangential axis 
20 34. Convergent axis 

36. Divergent axis 

FIG. 3 
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38. Fisheyelens 
40. Wider front 
42. Drum surface 
FIG. 4 
5 44. Weighted wheel 
46. Axis of rotation 
48. Open center 
50. Camera Operator 
52. Handle 
10 54. U-joint swivel 

56. Slip ring connection 
58. Input and output line 
FIG. 5 

60. First rotary position of Left sensor 
15 62. Second rotary position 

64. Third rotary position 

66. Fourth rotary position 

68. Fifth rotary position 

70. Near object 
20 72. Area of first sweeping scan 

74. Area of second sweeping scan 

76. Target area of near object 
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78. Axis for first position of Right sensor 
80. First position of Right sensor 
82. Scanned area on object from Right sensor 
84. Scanned area on object from Left sensor 
5 86. Area of overlapping scans 
88. Rangefinder emitter 
90. Rangefinder sensor 

FIG. 6 

it** ___ 

. .*\ 

It 92. Hemispherical linear scan from right sensor 

H 

j ii 10 94. Second hemispherical scan from right sensor 

& 

□ 96. Hemispherical linear scan from left sensor 

« 98. Second hemispherical scan from left sensor 

Q 

W 100. Vertical boundary of hemispherical FOV 

hi 

E 

!:! FIG. 7 

15 102. First equirectangular view 
104. Second equirectangular view 
FIG. 8 

106. Frame of information 
108. Uncorrected scans 
20 110. Scans shifted to right (delay forward in time) 
1 12. Scans shifted to left (delay backward in time) 
FIG. 9 



37 



# 



114. Region of Interest 

116. Selected scans shifted forward in time 

1 1 8. Selected scans shifted backward in time 

FIG. 10 
5 120. Boundary of Drum 

122. Power and Drive for sensor 

124. Central Power 

126. Central Control 

128. Motion Control 
10 130. Raw digital image data 

132. Image compression 

134. I/O formatting 

136. Slip rings 

138. Rangefinder 
15 140. Distance Data 

142. Delays 

144. Multiplexing 

146. Recorder 

148. Playback 
20 150. Data Transmission 

152. Decompression 

154. Region of Interest Extraction 



156. Display of ROI 
FIG. 11 

158. Extra interpolated sensor #2 

160. Extra interpolated sensor #3 
5 162. Extra interpolated sensor #4 

164. Basic optical axis for sensor #1 (Left) 

166. Extra optical axis for #2 

168. Extra optical axis for #3 

170. Extra optical axis for #4 
10 172. Rotational distance between sensors 1 and 4 

174. Lens for sensor #1 

176. Lens for sensor #2 

178. Lens for sensor #3 

180. Lens for sensor #4 
15 FIG. 12 

182. Scan from #1 

184. Scan from #2 

186. Delay for #1 

188. Recorded position for scan #1 
20 190. Recorded position for scan #2 

FIG. 13 

192. Enclosure 
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194. Conventional lens 

196. Streamlined front element for conventional lens 
FIG. 14 

198. Interior rotor 
200. Rotor above 
202. Rotor below 
204. Surface ripples 
206. Outer rotors 
208. Rotor tip guard rail 
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frt 10 210. Scoop Channel Enclosure 



212. Clockwise rotation 
214. Axis of rotation 



^ FIG. 15 
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216. Transmission line for sensor information 
15 218. Linear display 

220. Amplification and processing of signal for display 

222. Reduction optics 

224. Film recording 

226. Supply reel 
20 228. Takeupreel 

FIG. 16 

230. Axis of rotation 
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232. Supply reel for film camera drum 

234. Upper portion of drum 

236. Film in upper portion 

238. Middle portion of film camera drum 

240. Film in lower portion 

242. Exposure point 

244. Lower portion of film camera drum 

246. Takeup reel for film camera drum 



